Papers with unsupervised parser
Unsupervised Discourse Constituency Parsing Using Viterbi EM (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing studies on unsupervised discourse parsing have shown that it is expensive, time-consuming, and sometimes highly ambiguous. |
| Approach: | They propose an unsupervised parsing algorithm using Viterbi EM with a margin-based criterion and initialization methods for Viterbia training of discourse constituents based on prior knowledge of text structures. |
| Outcome: | The proposed method outperforms fully supervised parsers in terms of performance and learning of discourse constituents. |
Heads-up! Unsupervised Constituency Parsing via Self-Attention Heads (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing approaches to analyze syntactic knowledge of pre-trained language models have been limited. |
| Approach: | They propose an unsupervised method that extracts constituency trees from PLM attention heads. |
| Outcome: | The proposed method outperforms existing approaches if no development set is present. |
Simple Hardware-Efficient PCFGs with Independent Left and Right Productions (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to scaling dense PCFGs to thousands of nonterminals have shown to be beneficial for unsupervised parsing, but they still perform poorly as a language model and as an unsupervised model. |
| Approach: | They propose a simple PCFG formalism with independent left and right productions that scales more effectively as a language model and as an unsupervised parser. |
| Outcome: | The proposed formalism scales better as a language model and as an unsupervised parser despite imposing a stronger independence assumption compared to low-rank approaches. |
Fast-R2D2: A Pretrained Recursive Neural Network based on Pruned CKY for Grammar Induction and Text Representation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Chart-based models have shown great potential in unsupervised grammar induction, running recursively and hierarchically, but requiring O(n3) time-complexity. |
| Approach: | They propose a model-guided pruning method that scales to large language model pretraining by introducing a heuristic pruning method. |
| Outcome: | The proposed method significantly improves grammar induction quality and achieves competitive results in downstream tasks. |
Unsupervised Parsing via Constituency Tests (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for unsupervised parsing rely on constituency tests . linguists can judge a sentence's grammatical validity by modifying it via some transformation . |
| Approach: | They propose a method for unsupervised parsing based on a constituency test . they specify a set of transformations and use an unsupervised neural acceptability model to make grammaticality decisions. |
| Outcome: | The proposed method achieves 62.8 F1 on the Penn Treebank test set, an improvement of 7.6 points over the previous best results. |